Image shooting device and method for shooting image

ABSTRACT

An image shooting device for an object includes: a first light irradiation element for irradiating first light; a second light irradiation element for irradiating second light; an image shooting element; and a controller. A first irradiation region of the first light irradiation element is partially overlapped with a second irradiation region of the second light irradiation element. The controller controls the image shooting element to shoot a non-irradiation image of the object when no light irradiation element irradiates light. The controller controls the image shooting element to shoot a first image of the object when the first light irradiation element irradiates the first light. The controller controls the image shooting element to shoot a second image of the object when the second light irradiation element irradiates the second light.

CROSS REFERENCE TO RELATED APPLICATION

This application is based on Japanese Patent Application No. 2008-293329filed on Nov. 17, 2008, the disclosure of which is incorporated hereinby reference.

FIELD OF THE INVENTION

The present invention relates to an image shooting device for shootingan image by irradiating light and receiving reflected light and a methodfor shooting an image.

BACKGROUND OF THE INVENTION

Recently, a driving assist system for improving safety, convenience andcomfort of a vehicle for a passenger is disclosed in, for example,JP-A-2006-231963. A technique for detecting a face orientation and/or adirection of an eye is used for the system in the vehicle. The faceorientation and the direction of the eye are detected based on a data ofan image of a face of a human, which is shot by a device. Specifically,with using the face orientation and the direction of the eye, the systemdetermines whether a driver takes his eye off a road. Further, thesystem estimates physiological state and psychological state of thedriver based on variation of feature positions in his face. Thus, thesafety, convenience and comfort for the passenger is improved with usinginformation about the inattentive driving, the physiological state andpsychological state of the driver.

Further, a shooting method for shooting a face of a human when nearinfrared light is irradiated on the face of the human and for shootingthe face of the human when the near infrared light is not irradiated onthe face is described in, for example, JP-A-2007-4448. In this case, adifference image between a shot image when the near infrared light isirradiated on the face and a shot image when the near infrared light isnot irradiated on the face is obtained. With using the difference image,the shot image is obtained with reducing influence of an external lightsuch as sunshine.

However, when the near infrared light is irradiated on face of a personwearing glasses, the near infrared light is reflected on a lens of theglasses and/or a frame of the glasses. Thus, a portion with highbrightness appears on the shot image.

To remove the high brightness portion on the image, various methods areproposed. For example, a brightness histogram of picture cells, whichprovide the shot image, is formed, so that some of picture cells havingbrightness equal to or larger than a predetermined threshold arespecified as a high brightness portion. Then, the brightness in the highbrightness portion is replaced with average brightness of peripherypicture cells around the high brightness portion so that actualbrightness of the high brightness portion is reproduced. This techniqueis disclosed in, for example, JP-A-2002-269545. Further, anothertechnique is disclosed in JP-A-H08-185503. This technique is such that afirst threshold for extracting a reflection image of an eye ball of ahuman is defined, and a second threshold for extracting a reflectionimage of glasses worn by the human is defined. Then, based on a shape ofa binarized image of a shot image processed with using the first andsecond thresholds, the reflection image from the glasses is removed.

However, when the techniques in JP-A-2002-269545 and JP-A-H08-185503 areused, and the first and second thresholds are set to be certain values,a picture cell not related to the reflection of the near infrared lightmay be detected, or the high brightness portion provided by reflectionof the near infrared light may not be detected. Thus, from a practicalstandpoint, reliability of detection of the high brightness portion isreduced.

Further, in the technique in JP-A-2002-269545, the brightness of thehigh brightness portion is replaced with the average brightness of theperiphery picture cells. Accordingly, the replaced picture cells mayprovide a blurry image, and/or the brightness of the image may beuneven. Thus, the reproducibility of the brightness of the highbrightness portion is low.

SUMMARY OF THE INVENTION

In view of the above-described problem, it is an object of the presentdisclosure to provide an image shooting device for shooting an imagewith high reliability for detection of a high brightness portion andhigh reproducibility of brightness of the high brightness portion. It isanother object of the present disclosure to provide a method forshooting an image with high reliability for detection of a highbrightness portion and high reproducibility of brightness of the highbrightness portion.

According to a first aspect of the present disclosure, an image shootingdevice for shooting an image of an object includes: a first lightirradiation element for irradiating first light; a second lightirradiation element for irradiating second light; an image shootingelement for shooting an image with received light; and a controller. Thefirst light irradiation element has a first irradiation region, in whichthe first light from the first light irradiation element passes, and thesecond light irradiation element has a second irradiation region, inwhich the second light from the second light irradiation element passes.The first irradiation region is partially overlapped with the secondirradiation region. The image shooting element is disposed at apredetermined position so that the image shooting element receives lightreflected on the object, the light being one of the first and secondlight The object is arranged in an overlapping region of the first andsecond irradiation regions. The controller controls the image shootingelement to shoot a first image of the object when the first lightirradiation element irradiates the first light. The controller storesthe first image. The controller controls the image shooting element toshoot a second image of the object when the second light irradiationelement irradiates the second light. The controller stores the secondimage.

In the above device, with using the first image and the second image,the high brightness portion is detected, and the image information atthe high brightness portion is reproduced without using a threshold.Thus, reliability of detection of the high brightness portion isimproved. Further, it is not necessary to replace the brightness at thehigh brightness portion with brightness of a periphery portion. Sincethe brightness at the high brightness portion is replaced withbrightness at the high brightness portion which is actually detected,reproducibility of the brightness at the high brightness portion isimproved.

According to a second aspect of the present disclosure, a method forshooting an image of an object includes: irradiating first light;irradiating second light; receiving light and shooting an image withusing received light with an image shooting element; controlling theimage shooting element to shoot a first image of the object when thefirst light is irradiated; storing the first image; controlling theimage shooting element to shoot a second image of the object when thesecond light is irradiated; and storing the second image. The firstlight passes in a first irradiation region, and the second light passesin a second irradiation region. The first irradiation region ispartially overlapped with the second irradiation region. The imageshooting element is arranged at a predetermined position so that theimage shooting element receives light reflected on the object, the lightbeing one of the first and second lights. The object is arranged in anoverlapping region of the first and second irradiation regions.

In the above method, with using the first image and the second image,the high brightness portion is detected, and the image information atthe high brightness portion is reproduced without using a threshold.Thus, reliability of detection of the high brightness portion isimproved. Further, it is not necessary to replace the brightness at thehigh brightness portion with brightness of a periphery portion. Sincethe brightness at the high brightness portion is replaced withbrightness at the high brightness portion which is actually detected,reproducibility of the brightness at the high brightness portion isimproved.

According to a third aspect of the present disclosure, an image shootingdevice for shooting an image of an object includes: a first lightirradiation element for irradiating first light; a second lightirradiation element for irradiating second light; an image shootingelement for shooting an image with received light; and a controller. Thefirst light irradiation element has a first irradiation region, in whichthe first light from the first light irradiation element passes, and thesecond light irradiation element has a second irradiation region, inwhich the second light from the second light irradiation element passes.The first irradiation region is partially overlapped with the secondirradiation region. The image shooting element is disposed at apredetermined position so that the image shooting element receives lightreflected on the object, the light being one of the first and secondlight. The object is arranged in an overlapping region of the first andsecond irradiation regions. The controller controls the image shootingelement to shoot a non-irradiation image of the object when no lightirradiation element irradiates light. The controller stores thenon-irradiation image. The controller controls the image shootingelement to shoot a first image of the object when the first lightirradiation element irradiates the first light. The controller storesthe first image. The controller controls the image shooting element toshoot a second image of the object when the second light irradiationelement irradiates the second light, and the controller stores thesecond image.

In the above device, with using the first image and the second image,the high brightness portion is detected, and the image information atthe high brightness portion is reproduced without using a threshold.Thus, reliability of detection of the high brightness portion isimproved. Further, since the brightness at the high brightness portionis replaced with brightness at the high brightness portion which isactually detected, reproducibility of the brightness at the highbrightness portion is improved. Further, a first difference imagebetween the first image and the non-irradiation image and/or a seconddifference image between the second image and the non-irradiation imageare generated, so that the first or second difference image has smallinfluence of the external light other than the near infrared light.

According to a fourth aspect of the present disclosure, a method forshooting an image of an object includes: irradiating first light;irradiating second light; receiving light and shooting an image withusing received light with an image shooting element; controlling theimage shooting element to shoot a non-irradiation image of the objectwhen no light is irradiated; storing the non-irradiation image;controlling the image shooting element to shoot a first image of theobject when the first light is irradiated; storing the first image;controlling the image shooting element to shoot a second image of theobject when the second light is irradiated; and storing the secondimage. The first light passes in a first irradiation region, and thesecond light passes in a second irradiation region. The firstirradiation region is partially overlapped with the second irradiationregion. The image shooting element is arranged at a predeterminedposition so that the image shooting element receives light reflected onthe object, the light being one of the first and second lights. Theobject is arranged in an overlapping region of the first and secondirradiation regions.

In the above method, with using the first image and the second image,the high brightness portion is detected, and the image information atthe high brightness portion is reproduced without using a threshold.Thus, reliability of detection of the high brightness portion isimproved. Further, since the brightness at the high brightness portionis replaced with brightness at the high brightness portion which isactually detected, reproducibility of the brightness at the highbrightness portion is improved. Further, a first difference imagebetween the first image and the non-irradiation image and/or a seconddifference image between the second image and the non-irradiation imageare generated, so that the first or second difference image has smallinfluence of the external light other than the near infrared light.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the presentinvention will become more apparent from the following detaileddescription made with reference to the accompanying drawings. In thedrawings:

FIG. 1 is a block diagram showing an image shooting device;

FIG. 2 is a diagram showing an arrangement of a camera and near infraredlight irradiation elements;

FIG. 3 A is a top plan view showing a shooting range of the camera andan irradiation range of the near infrared light irradiation elements,and FIG. 3B is a side view showing the shooting range of the camera andthe irradiation range of the near infrared light irradiation elements;

FIG. 4 is a flowchart showing a reflection light removing process;

FIG. 5 is a diagram showing an image data obtained from a shot image ofa face of a driver;

FIGS. 6A to 6E are diagrams showing various image data obtained from theshot image of the face of the driver;

FIGS. 7A, 7C, 7E, 7G, 7I and 7K are photographs showing the face of thehuman wearing large glasses and facing a front side, and FIGS. 7B, 7D,7F, 7H, 7J and 7L are diagrams corresponding to the photographs in FIGS.7A, 7C, 7E, 7G, 7I and 7K, respectively;

FIGS. 8A, 8C, 8E, 8G, 8I and 8K are photographs showing the face of thehuman wearing small glasses and facing a front side, and FIGS. 8B, 8D,8F, 8H, 8J and 8L are diagrams corresponding to the photographs in FIGS.8A, 8C, 8E, 8G, 8I and 8K, respectively;

FIGS. 9A, 9C, 9E, 9G, 9I and 9K are photographs showing the face of thehuman wearing large glasses and facing obliquely, and FIGS. 9B, 9D, 9F,9H, 9J and 9L are diagrams corresponding to the photographs in FIGS. 9A,9C, 9E, 9G, 9I and 9K, respectively;

FIG. 10 is a time chart for explaining a method for generating a thirddifference image data according to a first embodiment;

FIG. 11 is a flowchart showing an irradiation and shooting process;

FIG. 12 is a flowchart showing a difference image generating process;

FIG. 13 is a flowchart showing a reflection light removed image process;

FIG. 14 is a time chart for explaining a method for generating a thirddifference image data according to a second embodiment;

FIG. 15A is a top plan view showing a shooting range of the camera andan irradiation range of the near infrared light irradiation elementsaccording to other embodiments, and FIG. 15B is a side view showing theshooting range of the camera and the irradiation range of the nearinfrared light irradiation elements;

FIG. 16A is a top plan view showing a shooting range of the camera andan irradiation range of the near infrared light irradiation elementsaccording to other embodiments, and FIG. 16B is a side view showing theshooting range of the camera and the irradiation range of the nearinfrared light irradiation elements;

FIG. 17A is a top plan view showing an irradiation direction of the nearinfrared light irradiation elements according to other embodiments, andFIGS. 17B and 17C are diagrams showing first and second high brightnessreflection positions in images; and

FIGS. 18A and 18B are a top plan view and a side view showing lightpaths of reflection light.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment

A first embodiment of the present disclosure will be explained asfollows with reference to the drawings.

FIG. 1 shows an image shooting device 1. FIG. 2 shows a view from adriver seat of a vehicle toward a front of the vehicle. Specifically,FIG. 2 shows an arrangement of a camera 2, and a pair of near infraredlight irradiation elements 4, 5.

The image shooting device 1 is mounted on the vehicle. The device 1includes the camera 2, an image capture board 3, a pair of near infraredlight irradiation elements 4, 5 and a controller 6. The camera 2continuously shoots a face of a person sitting on a driver seat, who isa driver DR of the vehicle. The image capture board 3 temporally storesdata of an image shot by the camera 2. The near infrared lightirradiation elements 4, 5 irradiates near infrared light toward the faceof the driver DR. The controller 6 executes an image processing withusing the image shot by the camera 2. Further, the controller 6 controlsthe camera 2 and the pair of the near infrared light irradiationelements 4, 5.

The camera 2 is a conventional camera having an image sensor, a lens andthe like. The image sensor includes multiple solid-state image sensingelements, which are arranged in a two-dimensional lattice pattern. Here,the image sensor may be a CCD image sensor or a CMOS image sensor. Thelens collects or focuses light from an object on the image sensor.

The image data obtained from the camera 2 is defined as a shot imagedata. The image data includes multiple picture cells, which are alignedin a two dimensional array having horizontal lines and vertical lines.The number of horizontal lines is M, and the number of vertical lines isN. Here, the number M and N are positive integer number. Thus, the shotimage data is composed of image cell data P(X, Y). The number of imagecell data is calculated by multiplying M by N. X represents a coordinateof the picture cell in the horizontal direction, and Y represents acoordinate of the picture cell in the vertical direction. Thus, X is oneof a group consisting of 0, 1, 2, . . . , and M. Y is one of a groupconsisting of 0, 1, 2, . . . , and N. The image cell data P(X, Y)represents the brightness of the picture cell arranged at a position of(X, Y). The brightness of the picture cell is defined as one of a groupconsisting of 0, 1, 2, . . . , and 255 so that the image cell data P(X,Y) is defined as one of 256 levels.

The camera 2 is arranged near an instrumental panel IP so that thecamera 2 shoots the face of the driver DR.

Each near infrared light irradiation element 4, 5 includes a LED lightsource for irradiating the near infrared light having a wavelength of850 nm, for example. The near infrared light irradiation element 4, 5 isarranged neat the instrumental panel IP so that the near infrared lightirradiation element 4, 5 emits the near infrared light toward the faceof the driver DR.

FIG. 3A shows a upper plan view of the camera 2 and the near infraredlight irradiation elements 4, 5 in order to show an irradiation area ofthe near infrared light irradiation elements 4, 5 and a shooting rangearea of the camera 2.

FIG. 3B shows a side view of the camera 2 and the near infrared lightirradiation elements 4, 5.

The first near infrared light irradiation element 4 has a firstirradiation region R1, which is surrounded with a line L1 and a line L2in FIG. 3A, so that the near infrared light irradiated from the firstnear infrared light irradiation element 4 reaches in the firstirradiation region R1. The second near infrared light irradiationelement 5 has a second irradiation region R2, which is surrounded with aline L3 and a line L4 in FIG. 3A, so that the near infrared lightirradiated from the second near infrared light irradiation element 5reaches in the second irradiation region R2. Thus, the first irradiationregion R1 is partially overlapped with the second irradiation region R2.

The overlapped irradiation region R3 is obtained from an overlappingregion of the first and second irradiation regions R1 and R2. The firstand second near infrared light irradiation elements 4, 5 irradiates thenear infrared light such that a whole of the head of the driver DR isdisposed in the overlapped irradiation region R3. Thus, the overlappedirradiation region R3 is surrounded with the line L5 and the line L6 inFIG. 3B. The head of the driver DR is arranged between the line L5 andthe line L6.

The camera 2 is arranged to shoot the whole of the face of the driverDR. Specifically, the camera 2 has a shooting range area, which issurrounded with the line L11 and the line L12 in the horizontaldirection. Further, the shooting range area is surrounded with the lineL13 and line L14 in the vertical direction. Further, the camera 2 isarranged on a line connecting between the arrangement point of the firstnear infrared light irradiation element 4 and the arrangement point ofthe second near infrared light irradiation element 5. Further, thecamera 2 is arranged at a middle point between the arrangement point ofthe first near infrared light irradiation element 4 and the arrangementpoint of the second near infrared light irradiation element 5.

The controller 6 is a conventional micro computer that includes a CPU, aROM, a RAM, a I/O device and a bus line for connecting among the CPU,the ROM the RAM and the I/O device. Based on a program stored in the ROMand/or the RAM, the controller 6 executes various processes.

In the image shooting device 1, the controller 1 executes to shoot theface of the driver DR and executes a reflection light removing processfor removing the influence of the reflection light from the shot image.

The controller 6 of the image shooting device 1 executes the reflectionlight removing process in FIG. 4. The reflection light removing processis repeatedly executed when the controller 6 turns on, i.e., when thecontroller is active.

When the reflection light removing process is executed, the controller 6determines whether a first shooting determination timer value is equalto or larger than a predetermined shooting interval determination value,which is for example, 16.6 milliseconds. Here, the first shootingdetermination timer value is set in the first shooting determinationtimer TM1, which is a timer that increments the value automatically ateach period such as one microsecond. When the first shootingdetermination timer value is reset to be zero at a certain time, thefirst shooting determination timer increments the first shootingdetermination timer value from zero since that time.

When the first shooting determination timer value is smaller than theshooting interval determination value, i.e., when the determination inStep S10 is “NO,” Step S10 is repeated. When the first shootingdetermination timer value is equal to or larger than the shootinginterval determination value, i.e., when the determination in Step S10is “YES,” in Step S20, the first shooting determination timer value isreset to be zero. Then, in Step S30, an image is shot by the camera 2,and data of the image shot by the camera 2 is generated. The data of theimage is defined as a shot image data. The shot image data generated inStep S30 is defined as a non-irradiation shot image data CD1. Then, inStep S40, the non-irradiation shot image data CD1 is stored in the RAMof the controller 6.

Then, in Step S50, the controller 6 determines whether the firstshooting determination timer value is equal to or larger than theshooting interval determination value. When the first shootingdetermination timer value is smaller than the shooting intervaldetermination value, i.e., when the determination in Step S50 is “NO,”Step S50 is repeated. When the first shooting determination timer valueis equal to or larger than the shooting interval determination value,i.e., when the determination in Step S50 is “YES,” in Step S60, thefirst shooting determination timer value is reset to be zero. Then, inStep S70, the controller 6 controls the first near infrared lightirradiation element 4 to start irradiating the near infrared light.

Then, in Step S80, an image is shot by the camera 2, and data of theimage shot by the camera 2 is generated. The data of the image isdefined as a shot image data. The shot image data generated in Step S80is defined as a first irradiation shot image data CD2. Then, in StepS90, the first irradiation shot image data CD2 is stored in the RAM ofthe controller 6.

Then, in Step S100, the controller 6 controls the first near infraredlight irradiation element 4 to stop irradiating the near infrared light.In Step S110, difference between brightness of the non-irradiation shotimage data CD1 obtained in Step S30 and brightness of the firstirradiation shot image data CD2 obtained in Step S80 is calculated sothat an image data as a first difference image data SD1 is obtained.

Here, the calculation of the difference between the brightness of thenon-irradiation shot image data CD1 and the brightness of the firstirradiation shot image data CD2 means that the brightness of the picturecell data Pn (X, Y) providing the non-irradiation shot image data CD1and the brightness of the picture cell data Pa (X, Y) providing thefirst irradiation shot image data CD2 are subtracted from each other.Here, the picture cell data Pn (X, Y) and the picture cell data Pa (X,Y) have the same horizontal coordinate and the vertical coordinate.

Specifically, when the horizontal coordinate in the first differenceimage data SD1 is X, and the vertical coordinate in the first differenceimage data SD1 is Y, the first difference image data SD1 is defined asPc (X, Y). The first difference image picture cell data Pc (X, Y) iscalculated from the formula F1.

Pc(X,Y)=Pa(X,Y)−Pn(X,Y)  (F1)

Here, when the first difference image picture cell data Pc (X, Y) issmaller than zero, the first difference image picture cell data Pc (X,Y) is set to be zero, so that deterioration of the difference imagecaused by noise or the like is prevented.

Then, in Step S120, the controller 6 determines whether the firstshooting determination timer value is equal to or larger than theshooting interval determination value. When the first shootingdetermination timer value is smaller than the shooting intervaldetermination value, i.e., when the determination in Step S120 is “NO,”Step S120 is repeated. When the first shooting determination timer valueis equal to or larger than the shooting interval determination value,i.e., when the determination in Step S120 is “YES,” in Step S130, thefirst shooting determination timer value is reset to be zero. Then, inStep S140, the controller 6 controls the second near infrared lightirradiation element 5 to start irradiating the near infrared light.

Then, in Step S150, an image is shot by the camera 2, and data of theimage shot by the camera 2 is generated. The data of the image isdefined as a shot image data. The shot image data generated in Step S150is defined as a second irradiation shot image data CD3. Then, in StepS160, the second irradiation shot image data CD3 is stored in the RAM ofthe controller 6.

Then, in Step S170, the controller 6 controls the second near infraredlight irradiation element 5 to stop irradiating the near infrared light.In Step S180, difference between brightness of the non-irradiation shotimage data CD1 obtained in Step S30 and brightness of the secondirradiation shot image data CD3 obtained in Step S150 is calculated sothat an image data as a second difference image data SD2 is obtained.

Here, the calculation of the difference between the brightness of thenon-irradiation shot image data CD1 and the brightness of the secondirradiation shot image data CD3 means that the brightness of the picturecell data Pn (X, Y) providing the non-irradiation shot image data CD1and the brightness of the picture cell data Pb (X, Y) providing thesecond irradiation shot image data CD3 are subtracted from each other.Here, the picture cell data Pn (X, Y) and the picture cell data Pb (X,Y) have the same horizontal coordinate and the vertical coordinate.

Specifically, when the horizontal coordinate in the second differenceimage data SD2 is X, and the vertical coordinate in the seconddifference image data SD2 is Y, the second difference image data SD2 isdefined as Pd (X, Y). The second difference image picture cell data Pd(X, Y) is calculated from the formula F2.

Pd(X,Y)=Pb(X,Y)−Pn(X,Y)  (F2)

Here, when the second difference image picture cell data Pd (X, Y) issmaller than zero, the second difference image picture cell data Pd (X,Y) is set to be zero, so that deterioration of the difference imagecaused by noise or the like is prevented.

In Step S190, the first difference image data SD1 and the seconddifference image data SD2 are added to each other so that an additionimage data AD1 is obtained.

Specifically, when the horizontal coordinate in the addition image dataAD1 is X, and the vertical coordinate in the addition image data AD1 isY, the addition image data AD1 is defined as Pe (X, Y). The additionimage picture cell data Pe (X, Y) is calculated from the formula F3.

Pe(X,Y)=Pc(X,Y)+Pd(X,Y)  (F3)

Further, in Step S200, an absolute value of difference betweenbrightness of the first difference image data SD1 obtained in Step S110and the brightness of the second difference image data SD2 obtained inStep S180 is calculated so that an image data as an absolute value ofdifference image data SD3 is obtained.

Specifically, when the horizontal coordinate in the absolute value ofdifference image data SD3 is X, and the vertical coordinate in theabsolute value of difference image data SD3 is Y, the absolute value ofdifference image data SD3 is defined as Pf (X, Y). The absolute value ofdifference image picture cell data Pf (X, Y) is calculated from theformula F4.

Pf(X,Y)=|Pc(X,Y)−Pd(X,Y)|  (F4)

Further, in Step S210, difference between brightness of the additionimage data AD1 obtained in Step S190 and the brightness of the absolutevalue of difference image data SD3 obtained in Step S200 is calculatedso that an image data as a third difference image data SD4 is obtained.

Specifically, when the horizontal coordinate in third difference imagedata SD4 is X, and the vertical coordinate in the third difference imagedata SD4 is Y, the third difference image data SD4 is defined as Pg (X,Y). The third difference image picture cell data Pg (X, Y) is calculatedfrom the formula F5.

Pg(X,Y)=Pe(X,Y)−Pf(X,Y)  (F5)

Here, when the third difference image picture cell data Pg (X, Y) issmaller than zero, the third difference image picture cell data Pg (X,Y) is set to be zero, so that deterioration of the difference imagecaused by noise or the like is prevented.

After Step S210, the reflection light removing process ends. In theimage shooting device 1, the camera 2 is disposed in a certain positionso that the camera 2 can receive the light reflected on the head of thedriver DR as a shooting object, which is arranged in the overlappedirradiation region R3. The camera 2 can shoot the shooting object whenthe first near infrared light irradiation element 4 irradiates the nearinfrared light on the shooting object, and further shoot the object whensecond near infrared light irradiation element 5 irradiates the nearinfrared light on the shooting object.

Further, the first irradiation region R1 and the second irradiationregion R2 are partially overlapped. Specifically, the first irradiationregion R1 is not completely overlapped on the second irradiation regionR2. i.e., the first irradiation region R1 is not the same as the secondirradiation region R2. Accordingly, when the first near infrared lightirradiation element 4 irradiates the near infrared light on the shootingobject, a first high brightness reflection position LP1 is defined onthe shooting object. The first high brightness reflection position LP1provides high brightness when the near infrared light from the firstnear infrared light irradiation element 4 is reflected on the shootingobject. When the second near infrared light irradiation element 5irradiates the near infrared light on the shooting object, a second highbrightness reflection position LP2 is defined on the shooting object.The second high brightness reflection position LP2 provides highbrightness when the near infrared light from the second near infraredlight irradiation element 5 is reflected on the shooting object. Thefirst high brightness reflection position LP1 is different from thesecond high brightness reflection position LP2 since the firstirradiation region R1 is not the same as the second irradiation regionR2.

Thus, image information at the first high brightness reflection positionLP1 in the first difference image data SD1 obtained in Step S110 islost. However, image information at the first high brightness reflectionposition LP1 in the second difference image data SD2 obtained in StepS180 is not lost. Similarly, image information at the second highbrightness reflection position LP2 in the second difference image dataSD2 obtained in Step S180 is lost. However, image information at thesecond high brightness reflection position LP2 in the first differenceimage data SD1 obtained in Step S110 is not lost.

Accordingly, the image information at the high brightness portion can bereproducible, and the image having the image information at the highbrightness portion, which is reproduced, is obtained according to thefollowing procedure.

First, difference between the first difference image data SD1 and thesecond difference image data SD2 is calculated so that a twicedifference image data is obtained.

Here, the brightness at the first high brightness reflection positionLP1 in the first difference image data SD1 is high. However, thebrightness at the first high brightness reflection position LP1 in thesecond difference image data SD2 is not high. Similarly, the brightnessat the second high brightness reflection position LP2 in the seconddifference image data SD2 is high. However, the brightness at the secondhigh brightness reflection position LP2 in the first difference imagedata SD1 is not high. Further, the brightness at the other portion ofthe first and second high brightness reflection positions LP1, LP2 inthe first difference image data SD1 is substantially the same as thebrightness at the other portion of the first and second high brightnessreflection positions LP1, LP2 in the second difference image data SD2.Specifically, the brightness of overlapping portion of the first andsecond difference image data SD1, SD2 in the first difference image dataSD1 is the same as the brightness of overlapping portion of the firstand second difference image data SD1, SD2 in the second difference imagedata SD2.

In this case, the twice difference image data provides the highbrightness at the first and second high brightness reflection positionsLP1, LP2 and the zero brightness at the other portions of the first andsecond high brightness reflection positions LP1, LP2, which are disposedin the overlapping portion of the first and second difference image dataSD1, SD2.

Specifically, the brightness at the first and second high brightnessreflection positions LP1, LP2 in the twice difference image data ishigh. The brightness at the other portions of the first and second highbrightness reflection positions LP1, LP2 in the twice difference imagedata is zero. Accordingly, the high brightness portion is easilydetected.

Accordingly, it is not necessary to use a threshold for detecting thehigh brightness portion. Thus, the image shooting device 1 preventsdetecting a portion other than the high brightness portion as the highbrightness portion in a case where the threshold is a certain value, andfurther, prevents not detecting the high brightness portion in a casewhere the threshold is another certain value. Accordingly, thereliability for detecting the high brightness portion is improved.

Next, the first difference image data SD1 is compared with the twicedifference image data. Thus, the high brightness portion in the firstdifference image data SD1 is specified. Further, the second differenceimage data SD2 is compared with the twice difference image data. Thus,the high brightness portion in the second difference image data SD2 isspecified.

Then, the image information at the first high brightness reflectionposition LP1 in the first difference image data SD1 is replaced with theimage information at the first high brightness reflection position LP1in the second difference image data SD2. Alternatively, the imageinformation at the second high brightness reflection position LP2 in thesecond difference image data SD2 is replaced with the image informationat the second high brightness reflection position LP2 in the firstdifference image data SD1. Thus, the brightness of the high brightnessportion is reproduced.

Thus, it is not necessary to replace the brightness of the highbrightness portion with the brightness of the periphery portion.Instead, the brightness of the high brightness portion is replaced withthe image information of the high brightness portion, which is actuallyobtained from the shot image. Accordingly, the reproducibility of thebrightness of the high brightness portion is improved.

The light emitted from the near infrared light irradiation elements 4, 5is the near infrared light. When both of the near infrared lightirradiation elements 4, 5 do not emit the light, the camera 2 shoots theface of the driver DR so that the non-irradiation shot image data CD1 isgenerated in Step S30.

Accordingly, the non-irradiation shot image data CD1 may includeinfluence of light other than the near infrared light, which is externalambient light such as sunshine. However, the non-irradiation shot imagedata CD1 does not include the influence of the near infrared light.Accordingly, the first irradiation shot image data CD2 generated in StepS80, the second irradiation shot image data CD3 generated in Step S150,the first difference image data SD1 obtained from the difference betweenthe non-irradiation shot image data CD1 and the first irradiation shotimage data CD2, and the second difference image data SD2 obtained fromthe difference between the non-irradiation shot image data CD1 and thesecond irradiation shot image data CD3 are generated, so that the shotimage data with reducing the influence of the light other than the nearinfrared light is obtained.

The reflection light removing process in FIG. 4 provides the image, inwhich the high brightness portion is reproduced. This process will beexplained with reference to FIGS. 5 and 6.

FIG. 5 shows the non-irradiation shot image data CD1, the firstirradiation shot image data CD2, the second irradiation shot image dataCD3, the first difference image data SD1, and the second differenceimage data SD2, which are obtained from images of the face of the driverDR wearing the glasses. FIG. 6 shows the first difference image dataSD1, and the second difference image data SD2, the addition image dataAD1, the absolute value of difference image data SD3, and the thirddifference image data SD4, which are obtained from processing the imagesof the face of the driver DR wearing the glasses.

The non-irradiation shot image data CD1 is obtained from the image,which is shot under a condition that the pair of near infrared lightirradiation elements 4, 5 do not emit the near infrared light. As shownin FIG. 5, the non-irradiation shot image data CD1 represents the imagehaving the low brightness of a whole of the face. Further, the firstirradiation shot image data CD2 represents the image, which is shotunder a condition that the first near infrared light irradiation element4 irradiates the near infrared light on the face of the driver DR from adiagonally forward right side of the driver DR. Thus, the brightness ofthe left periphery of the face is low, and the brightness of otherportion of the image is comparatively high. The second irradiation shotimage data CD3 represents the image, which is shot under a conditionthat the second near infrared light irradiation element 5 irradiates thenear infrared light on the face of the driver DR from a diagonallyforward left side of the driver DR. Thus, the brightness of the rightperiphery of the face is low, and the brightness of other portion of theimage is comparatively high.

Thus, the first difference image data SD1, which is obtained bysubtracting the brightness of the first irradiation shot image data CD2from the brightness of the non-irradiation shot image data CD1,represents the image of the face, from which a part of the imageproviding the left periphery of the face is removed. The seconddifference image data SD2, which is obtained by subtracting thebrightness of the second irradiation shot image data CD3 from thebrightness of the non-irradiation shot image data CD1, represents theimage of the face, from which a part of the image providing the rightperiphery of the face is removed.

Thus, the part of the image providing the left periphery of the face isremoved from the image of the first difference image data SD1, and thepart of the image is shot by receiving light other than the nearinfrared light reflected on the left periphery. The part of the imageproviding the right periphery of the face is removed from the image ofthe second difference image data SD2, and the part of the image is shotby receiving light other than the near infrared light reflected on theright periphery.

Here, the first high brightness reflection position LP1 of the firstdifference image data SD1 is different from the second high brightnessreflection position LP2 of the second difference image data SD2.

As shown in FIGS. 6A and 6B, the brightness at the first high brightnessreflection position LP1 of the first difference image data SD1 isdefined as, for example, 230. The brightness at the other portions ofthe first difference image data SD1 is defined as, for example, 100. Thebrightness at the second high brightness reflection position LP2 of thesecond difference image data SD2 is defined as, for example, 230. Thebrightness at the other portions of the second difference image data SD2is defined as, for example, 100.

In this case, the addition image data AD1, which is obtained by addingthe first difference image data SD1 and the second difference image dataSD2, has the brightness of 330 at the first and second high brightnessreflection positions LP1, LP2, the brightness of 200 at a center portionof the face other than the first and second high brightness reflectionpositions LP1, LP2, and the brightness of 100 at the right and leftperipheries of the face, as shown in FIG. 6C.

Further, the absolute value of difference image data SD3, which is anabsolute value of difference between the brightness of the firstdifference image data SD1. and the brightness of the second differenceimage data SD2, has the brightness of 130 at the first and second highbrightness reflection positions LP1, LP2, the brightness of zero at thecenter portion of the face other than the first and second highbrightness reflection positions LP1, LP2, and the brightness of 100 atthe right and left peripheries of the face, as shown in FIG. 6D.

Thus, the third difference image data SD4, which is obtained bysubtracting the brightness of the absolute value of difference imagedata SD3 from the brightness of the addition image data AD1, has thebrightness of 200 at the first and second high brightness reflectionpositions LP1, LP2, the brightness of 200 at the center portion of theface other than the first and second high brightness reflectionpositions LP1, LP2, and the brightness of zero at the right and leftperipheries of the face, as shown in FIG. 6E.

The image shooting device 1 executes the reflection light removingprocess so that the following result is obtained.

FIG. 7A shows the non-irradiation shot image data CD1 obtained byshooting the face of the human HM, who faces a front side and wearslarge glasses. FIG. 7B is an illustrative diagram of the non-irradiationshot image data CD1 in FIG. 7A. FIG. 7C shows the first irradiation shotimage data CD2, and FIG. 7D is an illustrative diagram of firstirradiation shot image data CD2 in FIG. 7C. FIG. 7E shows the secondirradiation shot image data CD3, and FIG. 7F is an illustrative diagramof the second irradiation shot image data CD3 in FIG. 7E. FIG. 7G showsthe first difference image data SD1, and FIG. 7H is an illustrativediagram of the first difference image data SD1 in FIG. 7G. FIG. 7I showsthe second difference image data SD2, and FIG. 7J is an illustrativediagram of the second difference image data SD2 in FIG. 7J. FIG. 7Kshows the third difference image data SD4, and FIG. 7L is anillustrative diagram of the third difference image data SD4 in FIG. 7K.

In the non-irradiation shot image data CD1, the first high brightnessportion HB1 caused by the external light is disposed on an upper side ofthe human HM, and the background object BO is disposed on the back ofthe human HM.

In the first irradiation shot image data CD2, the first high brightnessportion HB1, the background object BO and the second high brightnessportion HB2 caused by the near infrared light emitted from the nearinfrared light irradiation element 4 are disposed.

In the second irradiation shot image data CD3, the first high brightnessportion HB1, the background object BO and the third high brightnessportion HB3 caused by the near infrared light emitted from the nearinfrared light irradiation element 5 are disposed.

In the first difference image data SD1, the first high brightnessportion HB1 and the background object BO are not disposed, and thesecond high brightness portion HB2 is disposed. Thus, the firstdifference image data SD1. provides the image, from which a part of theimage is removed, the part being shot by the external light other thanthe near infrared light and being the first high brightness portion HB1and the background object BO.

Similarly, in the second difference image data SD2, the first highbrightness portion HB1 and the background object BO are not disposed,and the third high brightness portion HB3 is disposed. Thus, the seconddifference image data SD2 provides the image, from which a part of theimage is removed, the part being shot by the external light other thanthe near infrared light and being the first high brightness portion HB1and the background object BO.

In the third difference image data SD4, the first to third highbrightness portions HB1-HB3 and the background object BO are notdisposed. Thus, the third difference image data SD4 provides the image,from which the part of the image is removed, the part being shot by theexternal light other than the near infrared light, and further, thesecond and third high brightness portions HB2-HB3 are reproduced in theimage.

FIG. 8A shows the non-irradiation shot image data CD1 obtained byshooting the face of the human HM, who faces a front side and wearssmall glasses. FIG. 8B is an illustrative diagram of the non-irradiationshot image data CD1 in FIG. 8A. FIG. 8C shows the first irradiation shotimage data CD2, and FIG. 8D is an illustrative diagram of firstirradiation shot image data CD2 in FIG. 8C. FIG. 8E shows the secondirradiation shot image data CD3, and FIG. 8F is an illustrative diagramof the second irradiation shot image data CD3 in FIG. 8E. FIG. 8G showsthe first difference image data SD1, and FIG. 8H is an illustrativediagram of the first difference image data SD1. in FIG. 8G. FIG. 8Ishows the second difference image data SD2, and FIG. 8J is anillustrative diagram of the second difference image data SD2 in FIG. 8J.FIG. 8K shows the third difference image data SD4, and FIG. 8L is anillustrative diagram of the third difference image data SD4 in FIG. 8K.

As shown in FIGS. 8A-8L, even when the human HM wears the small glasses,the second and third high brightness portions HB2-HB3 are reproduced inthe image of the third difference image data SD4.

FIG. 9A shows the non-irradiation shot image data CD1 obtained byshooting the face of the human HM, who faces obliquely and wears smallglasses. FIG. 9B is an illustrative diagram of the non-irradiation shotimage data CD1 in FIG. 9A. FIG. 9C shows the first irradiation shotimage data CD2, and FIG. 9D is an illustrative diagram of firstirradiation shot image data CD2 in FIG. 9C. FIG. 9E shows the secondirradiation shot image data CD3, and FIG. 9F is an illustrative diagramof the second irradiation shot image data CD3 in FIG. 9E. FIG. 9G showsthe first difference image data SD1, and FIG. 9H is an illustrativediagram of the first difference image data SD1 in FIG. 9G. FIG. 9I showsthe second difference image data SD2, and FIG. 9J is an illustrativediagram of the second difference image data SD2 in FIG. 9J. FIG. 9Kshows the third difference image data SD4, and FIG. 9L is anillustrative diagram of the third difference image data SD4 in FIG. 9K.

As shown in FIGS. 9A-9L, even when the human HM faces obliquely andwears the large glasses, the second and third high brightness portionsHB2-HB3 are reproduced in the image of the third difference image dataSD4.

FIG. 10 shows a time chart for explaining a generating method of thethird difference image data SD4.

The camera 2 shoots images of the face of the human HM at everypredetermined time intervals Tc. The first near infrared lightirradiation element 4 emits the near infrared light at every triplicatetime intervals 3Tc of shooting time interval Tc so as to coincide withtime at which the camera 2 shoots an image of the face, The second nearinfrared light irradiation element 5 emits the near infrared light atevery triplicate time intervals 3Tc of shooting time interval Tc so asto coincide with time at which the camera 2 shoots an image of the faceafter the shooting time interval Tc has elapsed from the irradiation ofthe first near infrared light irradiation element 4.

Accordingly, the image shooting device 1 generates the non-irradiationshot image data CD1, the first irradiation shot image data CD2, and thesecond irradiation shot image data CD3 at every shooting time intervalTc in this order.

The image shooting device 1 generates the first difference image dataSD1 after each one of the first irradiation shot image data CD2 isgenerated. Further, the image shooting device 1 generates the seconddifference image data SD2 after each one of the second irradiation shotimage data CD3 is generated.

Further, the image shooting device 1 generates the third differenceimage data SD4 after each one of the second difference image data SD2 isgenerated. Thus, the image shooting device 1 generates the thirddifference image data SD4 at every triplicate time intervals 3Tc ofshooting time interval Tc.

The camera 2 is disposed at a middle point between the arrangementposition of the first near infrared light irradiation element 4 and thearrangement position of the second near infrared light irradiationelement 5. Accordingly, compared with a case where the camera 2 isarranged near the arrangement position of the first or second nearinfrared light irradiation element 4, 5, difference between a first shotimage and a second shot image is reduced. Here, the first shot image isdefined as an image provided by the first irradiation shot image dataCD2, and the second shot image is defined as an image provided by thesecond irradiation shot image data CD3. Thus, an image of the shootingobject in the first shot image is largely overlapped with an image ofthe shooting object in the second shot image. Accordingly, a portion ofthe image shot by the camera 2, in which the brightness of the highbrightness portion is reproducible, is large.

The image shooting device 1 is mounted on the vehicle, so that thedevice 1 shoots the face of the driver DR, who drives the vehicle. Thereproducibility of the high brightness portion in the image of the faceof the driver is high. Accordingly, even if a part of the imagecorresponding to an eye of the driver has high brightness, the imageinformation of the eye of the driver is reproduced in the image of theface of the driver. Accordingly, detection accuracy of a faceorientation and/or a direction of an eye are/is improved.

In the present embodiment, the first near infrared light irradiationelement 4 provides a first light emitting element, and the second nearinfrared light irradiation element 5 provides a second light emittingelement. The camera provides a shooting device. The process in Steps S70and S100 provide a first irradiation control element and procedure. Theprocess in Step S80 provides a first image obtaining element andprocedure. The process in Steps S140 and S170 provide a secondirradiation control element and procedure. The process in Step S150provides a second image obtaining element and procedure.

The process in Step S190 provides an image adding element and procedure.The process in Step S200 provides a first image difference element andprocedure. The process in Step S210 provides a second image differenceelement and procedure. The process in Step S30 provides a third imageobtaining element and procedure. The first irradiation shot image dataCD2 provides a first shot image, and the second irradiation shot imagedata CD3 provides a second shot image. The addition image data AD1provides an adding image. The absolute value of difference image dataSD3 provides a first difference image, and the third difference imagedata SD4 provides a second difference image.

Second Embodiment

The image shooting device 1 according to a second embodiment executesindependently an irradiation and shooting process, a difference imagegenerating process and a reflection light removed image process, insteadof the reflection light removing process. The irradiation and shootingprocess provides to control the irradiation time of the near infraredlight and the shooting time of the camera 2. The difference imagegenerating process provides to generate the first difference image dataSD1 and the second difference image data SD2. The reflection lightremoved image process provides to generate an image, from which theinfluence of the reflection light is removed.

The irradiation and shooting process executed by the controller 6 in thedevice 1 will be explained with reference to FIG. 11. FIG. 11 is aflowchart of the irradiation and shooting process. The irradiation andshooting process is repeated while the controller 6 turns on, i.e.,switches on.

When the irradiation and shooting process is executed, the controller 6determines in Step S310 whether a first shooting determination timervalue is equal to or larger than a predetermined shooting intervaldetermination value.

When the first shooting determination timer value is smaller than theshooting interval determination value, i.e., when the determination inStep S310 is “NO,” Step S310 is repeated. When the first shootingdetermination timer value is equal to or larger than the shootinginterval determination value, i.e., when the determination in Step S310is “YES,” in Step S320, the first shooting determination timer value isreset to be zero. Then, in Step S330, an image is shot by the camera 2,and data of the image shot by the camera 2 is generated. The data of theimage is defined as a shot image data. The shot image data generated inStep S330 is defined as a non-irradiation shot image data CD1. Then, inStep S340, the non-irradiation shot image data CD1 is stored in the RAMof the controller 6.

Then, in Step S350, the controller 6 determines whether the firstshooting determination timer value is equal to or larger than theshooting interval determination value. When the first shootingdetermination timer value is smaller than the shooting intervaldetermination value, i.e., when the determination in Step S350 is “NO,”Step S350 is repeated. When the first shooting determination timer valueis equal to or larger than the shooting interval determination value,i.e., when the determination in Step S350 is “YES,” in Step S360, thefirst shooting determination timer value is reset to be zero. Then, inStep S370, the controller 6 controls the first near infrared lightirradiation element 4 to start irradiating the near infrared light.

Then, in Step S380, an image is shot by the camera 2, and data of theimage shot by the camera 2 is generated. The data of the image isdefined as a shot image data. The shot image data generated in Step S380is defined as a first irradiation shot image data CD2. Then, in StepS390, the first irradiation shot image data CD2 is stored in the RAM ofthe controller 6. Further, in Step S400, the controller 6 controls thefirst near infrared light irradiation element 4 to stop irradiating thenear infrared light.

Then, in Step S410, the controller 6 determines whether the firstshooting determination timer value is equal to or larger than theshooting interval determination value. When the first shootingdetermination timer value is smaller than the shooting intervaldetermination value, i.e., when the determination in Step S410 is “NO,”Step S410 is repeated. When the first shooting determination timer valueis equal to or larger than the shooting interval determination value,i.e., when the determination in Step S410 is “YES,” in Step S420, thefirst shooting determination timer value is reset to be zero. Then, inStep S430, the controller 6 controls the second near infrared lightirradiation element 5 to start irradiating the near infrared light.

Then, in Step S440, an image is shot by the camera 2, and data of theimage shot by the camera 2 is generated. The data of the image isdefined as a shot image data. The shot image data generated in Step S440is defined as a second irradiation shot image data CD3. Then, in StepS450, the second irradiation shot image data CD3 is stored in the RAM ofthe controller 6. Then, in Step S460, the controller 6 controls thesecond near infrared light irradiation element 5 to stop irradiating thenear infrared light.

Next, the difference image generating process executed by the controller6 in the device 1 will be explained with reference to FIG. 12. FIG. 12is a flowchart of the difference image generating process. Thedifference image generating process is repeated while the controller 6turns on, i.e., switches on.

When the difference image generating process is executed, the controller6 determines whether a second shooting determination timer value isequal to or larger than a predetermined shooting interval determinationvalue. Here, the second shooting determination timer value is set in thesecond shooting determination timer TM2, which is a timer thatincrements the value automatically at each period such as onemicrosecond. When the second shooting determination timer value is resetto be zero at a certain time, the second shooting determination timerTM2 increments the second shooting determination timer value from zerosince that time. Further, the second shooting determination timer TM2starts to increment after a predetermined first delay time has elapsedfrom a time at which the first shooting determination timer TM1 startsto increment just after the controller 6 is activated.

Here, when the second shooting determination timer value is smaller thanthe shooting interval determination value, i.e., when the determinationin Step S510 is “NO,” Step S510 is repeated. When the second shootingdetermination timer value is equal to or larger than the shootinginterval determination value, i.e., when the determination in Step S510is “YES,” in Step S520, the second shooting determination timer value isreset to be zero. Then, in Step S530, difference between brightness ofthe non-irradiation shot image data CD1 obtained in Step S330 andbrightness of the first irradiation shot image data CD2 obtained in StepS380 is calculated so that an image data as a first difference imagedata SD1 is obtained.

Then, in Step S540, the controller 6 determines whether the secondshooting determination timer value is equal to or larger than theshooting interval determination value. When the second shootingdetermination timer value is smaller than the shooting intervaldetermination value, i.e., when the determination in Step S540 is “NO,”Step S540 is repeated. When the second shooting determination timervalue is equal to or larger than the shooting interval determinationvalue, i.e., when the determination in Step S540 is “YES,” in Step S550,the second shooting determination timer value is reset to be zero. Then,in Step S560, difference between brightness of the non-irradiation shotimage data CD1 obtained in Step S330 and brightness of the secondirradiation shot image data CD3 obtained in Step S440 is calculated sothat an image data as a second difference image data SD2 is obtained.Then, the difference image generating process ends,

Next, the reflection light removed image process executed by thecontroller 6 in the device 1 will be explained with reference to FIG.13. FIG. 13 is a flowchart of the reflection light removed imageprocess. The reflection light removed image process is repeated whilethe controller 6 turns on, i.e., switches on.

When the reflection light removed image process is executed, thecontroller 6 determines whether a third shooting determination timervalue is equal to or larger than a predetermined shooting intervaldetermination value. Here, the third shooting determination timer valueis set in the third shooting determination timer TM3, which is a timerthat increments the value automatically at each period such as onemicrosecond. When the third shooting determination timer value is resetto be zero at a certain time, the third shooting determination timer TMincrements the third shooting determination timer value from zero sincethat time. Further, the third shooting determination timer TM3 starts toincrement after a predetermined second delay time has elapsed from atime at which the second shooting determination timer TM2 starts toincrement just after the controller 6 is activated.

Here, when the third shooting determination timer value is smaller thanthe shooting interval determination value, i.e., when the determinationin Step S610 is “NO,” Step S610 is repeated. When the third shootingdetermination timer value is equal to or larger than the shootinginterval determination value, i.e., when the determination in Step S610is “YES,” in Step S620, the third shooting determination timer value isreset to be zero.

Then, in Step S630, the first difference image data SD1 obtained in StepS530 and the second difference image data SD2 obtained in Step S560 areadded to each other so that an addition image data AD1 is obtained.Further, in Step S640, an absolute value of difference betweenbrightness of the first difference image data SD1 obtained in Step S530and the brightness of the second difference image data SD2 obtained inStep S560 is calculated so that an image data as an absolute value ofdifference image data SD3 is obtained.

Then, in Step S650, difference between brightness of the addition imagedata AD1 obtained in Step S630 and the brightness of the absolute valueof difference image data SD3 obtained in Step S640 is calculated so thatan image data as a third difference image data SD4 is obtained. Thus,the reflection light removed image process ends.

FIG. 14 shows a time chart for explaining a generating method of thethird difference image data SD4.

The camera 2 shoots images of the face of the human HM at everypredetermined time intervals Tc. The first near infrared lightirradiation element 4 emits the near infrared light at every triplicatetime intervals 3Tc of shooting time interval Tc so as to coincide withtime at which the camera 2 shoots an image of the face. The second nearinfrared light irradiation element 5 emits the near infrared light atevery triplicate time intervals 3Tc of shooting time interval Tc so asto coincide with time at which the camera 2 shoots an image of the faceafter the shooting time interval Tc has elapsed from the irradiation ofthe first near infrared light irradiation element 4.

Accordingly, the image shooting device 1 generates the non-irradiationshot image data CD1, the first irradiation shot image data CD2, and thesecond irradiation shot image data CD3 at every shooting time intervalTc in this order.

The device 1 executes a process for generating the first differenceimage data SD1 and a process for generating the second difference imagedata SD2 alternately. The process for generating the first differenceimage data SD1 is performed with using a latest non-irradiation shotimage data CD1 and a latest first irradiation shot image data CD2. Theprocess for generating the second difference image data SD2 is performedwith using a latest non-irradiation shot image data CD1 and a latestsecond irradiation shot image data CD3.

The device 1 generates the third difference image data SD4 with using alatest first difference image data SD1 and a latest second differenceimage data SD2 after each one of the first difference image data SD1 andthe second difference image data SD2 is generated.

Thus, the image shooting device 1 generates the third difference imagedata SD4 at every shooting time intervals Tc.

In the embodiments, the device 1 shoots the face of the driver.Alternatively, the device 1 may shoot other objects. Specifically, whenit is necessary to remove the influence of the reflection light from theobject image, the device 1 provides effective results.

Although the device 1 shoots a whole of the face of the driver, thedevice 1 may shoot a part of the face around the eye of the driver.

In the embodiments, the camera 2 is arranged at a middle point betweenthe arrangement position of the first near infrared light irradiationelement 4 and the arrangement position of the second near infrared lightirradiation element 5 along with the line connecting between thearrangement position of the first near infrared light irradiationelement 4 and the arrangement position of the second near infrared lightirradiation element 5. Here, the line is defined as an irradiationelement connecting line. Further, the height of the arrangement positionof the camera 2 is the same as the height of the arrangement position ofeach of the first near infrared light irradiation element 4 and thesecond near infrared light irradiation element 5. Alternatively, thecamera 2 may be arranged between the arrangement position of the firstnear infrared light irradiation element 4 and the arrangement positionof the second near infrared light irradiation element 5.

For example, as shown in FIGS. 15A and 15B, although the camera 2 isarranged at a middle position between the first near infrared lightirradiation element 4 and the second near infrared light irradiationelement 5, the height of the arrangement position of the camera 2 isdifferent from the height of the arrangement position of each of thefirst near infrared light irradiation element 4 and the second nearinfrared light irradiation element 5.

Alternatively, as shown in FIGS. 16A and 16B, the height of thearrangement position of the first near infrared light irradiationelement 4 is different from the height of the arrangement position ofthe second near infrared light irradiation element 5. The camera 2 isarranged at a middle position between the first near infrared lightirradiation element 4 and the second near infrared light irradiationelement 5.

The irradiation direction of each of the first near infrared lightirradiation element 4 and the second near infrared light irradiationelement 5 may be different from the direction shown in FIGS. 3A, 3B, 16Aand 16B.

For example, as shown in FIG. 17A, the first near infrared lightirradiation element 4 irradiates the near infrared light toward the faceof the driver DR along with an irradiation direction, which is differentfrom an irradiation direction of the second near infrared lightirradiation element 5. In this case, as shown in FIGS. 17B and 17C, thefirst high brightness reflection position LP1 in the image of the firstirradiation shot image data CD2 is different from the second highbrightness reflection position LP2 in the image of the secondirradiation shot image data CD3.

Alternatively, as shown in FIG. 18A, the camera 2 may includes a lens 21and an image sensor 23 having multiple solid-state image sensingelements 22. The near infrared light emitted from the first nearinfrared light irradiation element 4 is reflected on the face of thedriver DR, and the near infrared light emitted from the second nearinfrared light irradiation element 5 is reflected on the face of thedriver DR. The reflected light RL1 from the first near infrared lightirradiation element 4 and the reflected light RL2 from the second nearinfrared light irradiation element 5 pass through the lens 21, and then,are concentrated on the image sensor 23. The solid-state image sensingelements 22 are arranged on the image sensor 23 with a two-dimensionallattice pattern.

As shown in FIG. 18B, a part of the reflected light RL1 corresponding tothe first high brightness reflection position LP1 reaches a firstposition AP1 on the image sensor 23. A part of the reflected light RL2corresponding to the second high brightness reflection position LP2reaches a second position AP2 on the image sensor 23. In this case, itis necessary to set the irradiation direction of each of the first andsecond near infrared light irradiation elements 4, 5 so as to separatethe first and second positions AP1, AP2 from each other by apredetermined distance SP equal to or larger than an arrangementdistance of the solid-state image sensing elements 22. This is becausethe first position AP1 corresponding to the first high brightnessreflection position LP1 does not overlap with the second position AP2corresponding to the second high brightness reflection position LP2.

Specifically, the irradiation direction of each of the first and secondnear infrared light irradiation elements 4, 5 is set such that the firstposition AP1 is spaced apart from the second position AP2 by one or morepicture cells.

In the embodiments, the third difference image data SD4 is generatedwith using the formulas F3-F5. Alternatively, the third difference imagedata SD4 may be generated with using a following formula F6 or F7.

Pg(X,Y)=Pa(X,Y)+Pb(X,Y)−|Pa(X,Y)−Pb(X,Y)|−2pn(X,Y)   F6

Pg(X,Y)=Pc(X,Y)+Pd(X,Y)−|Pa(X,Y)−Pb(X,Y)|  F7

Further, in the embodiments, the first difference image data SD1 and thesecond difference image data SD2 are generated with using the formulasF1-F2. Further, when the first difference image picture cell data Pc (X,Y) is smaller than zero, the first difference image picture cell data Pc(X, Y) is set to be zero. When the second difference image picture celldata Pd (X, Y) is smaller than zero, the second difference image picturecell data Pd (X, Y) is set to be zero. Alternatively, when the lightamount of the near infrared light emitted from each of the first andsecond near infrared light irradiation elements 4, 5 is sufficientlylarge, the first difference image data SD1 and the second differenceimage data SD2 may be generated with using following formulas F8 and F9.In this case, the first difference image picture cell data Pc (X, Y) andthe second difference image picture cell data Pd (X, Y) are always equalto or larger than zero. Accordingly, it is not necessary to execute aprocess for setting the first difference image picture cell data Pc (X,Y) to be zero when the first difference image picture cell data Pc (X,Y) is smaller than zero, and a process for setting the second differenceimage picture cell data Pd (X, Y) to be zero when the second differenceimage picture cell data Pd (X, Y) is smaller than zero.

Pc(X,Y)=|Pa(X,Y)−Pn(X,Y)|  F8

Pd(X,Y)=|Pb(X,Y)−Pn(X,Y)|  F9

Further, in the embodiments, the camera 2 and the first and second nearinfrared light irradiation elements 4, 5 are arranged near theinstrumental panel IP. Alternatively, the camera 2 and the first andsecond near infrared light irradiation elements 4, 5 may be arranged ondifferent place as long as the camera 2 shoots the face of the driver DRand the first and second near infrared light irradiation elements 4, 5irradiate the near infrared light toward the face of the driver DR. Forexample, the camera 2 and the first and second near infrared lightirradiation elements 4, 5 may be arranged near a steering column.

The device 1 may further include a display for displaying the image ofthe third difference image data SD4.

The above disclosure has the following aspects.

According to a first aspect of the present disclosure, an image shootingdevice for shooting an image of an object includes: a first lightirradiation element for irradiating first light; a second lightirradiation element for irradiating second light; an image shootingelement for shooting an image with received light; and a controller. Thefirst light irradiation element has a first irradiation region, in whichthe first light from the first light irradiation element passes, and thesecond light irradiation element has a second irradiation region, inwhich the second light from the second light irradiation element passes.The first irradiation region is partially overlapped with the secondirradiation region. The image shooting element is disposed at apredetermined position so that the image shooting element receives lightreflected on the object, the light being one of the first and secondlight. The object is arranged in an overlapping region of the first andsecond irradiation regions. The controller controls the image shootingelement to shoot a first image of the object when the first lightirradiation element irradiates the first light. The controller storesthe first image. The controller controls the image shooting element toshoot a second image of the object when the second light irradiationelement irradiates the second light. The controller stores the secondimage.

Here, the above device corresponds to the device according to the firstand second embodiment under a condition that the brightness of thenon-irradiation image is zero. Specifically, all of the non-irradiationshot image data CD1 in the non-irradiation image are zero.

In the above device, since the first irradiation region is partiallyoverlapped with the second irradiation region, the first irradiationregion is not the same as the second irradiation region. Accordingly, afirst high brightness portion of the object is different from a secondhigh brightness portion. When the first light irradiation elementirradiates the first light toward the object, and the first lightreflects on the object at the first high brightness portion so that thebrightness of the first high brightness portion becomes high. When thesecond light irradiation element irradiates the second light toward theobject, and the second light reflects on the object at the second highbrightness portion so that the brightness of the second high brightnessportion becomes high. Thus, the first image in the controller losesimage information at the first high brightness portion, and the secondimage in the controller loses image information at the second highbrightness portion. However, the second image in the controller hasimage information at the first high brightness portion, and the firstimage in the controller has image information at the second highbrightness portion.

Accordingly, with using the first image and the second image, the highbrightness portion is detected, and the image information at the highbrightness portion is reproduced without using a threshold. Thus,reliability of detection of the high brightness portion is improved.Further, it is not necessary to replace the brightness at the highbrightness portion with brightness of a periphery portion. Since thebrightness at the high brightness portion is replaced with brightness atthe high brightness portion which is actually detected, reproducibilityof the brightness at the high brightness portion is improved.

Alternatively, the controller may include a first light irradiationcontrol means, a first image storing means, a second light irradiationcontrol means and a second image storing means. The first lightirradiation control means controls the first light irradiation elementto irradiate only the first light. The first image storing meanscontrols the image shooting element to shoot the first image only whenthe first light irradiation element irradiates the first light. Thesecond light irradiation control means controls the second lightirradiation element to irradiate only the second light. The second imagestoring means controls the image shooting element to shoot the secondimage only when the second light irradiation element irradiates thesecond light.

Alternatively, the controller may generate an addition image by addingbrightness of the first image and brightness of the second image. Thecontroller may generate an absolute difference image by calculating anabsolute value of difference between the brightness of the first imageand the brightness of the second image. The controller may generate afinal difference image by subtracting brightness of the absolutedifference image from brightness of the addition image.

In the above case, the first image includes a first high brightnessportion with a first high brightness, which is defined as A1, and otherportions with a first normal brightness, which is defined as A2. Thesecond image includes a second high brightness portion with a secondhigh brightness, which is defined as B1, and other portions with asecond normal brightness, which is defined as B2. Here, the first highbrightness A1 is larger than the first normal brightness A2, and thesecond high brightness B1 is larger than the second normal brightnessB2. The first normal brightness A2 is the same as the second normalbrightness B2.

In this case, the brightness of the addition image at the first highbrightness portion is defined as C1, the brightness of the additionimage at the second high brightness portion is defined as C2, and thebrightness of the addition image at other portions is defined as C3.

Each brightness C1-C3 is provided by the following formulas F10-F12.

C1=A1+B2  F10

C2=A2+B1  F11

C3=A2+B2   F12.

Further, the brightness of the absolute difference image at the firsthigh brightness portion is defined as D1, the brightness of the absolutedifference image at the second high brightness portion is defined as D2,and the brightness of the absolute difference image at other portions isdefined as D3. Each brightness D1-D3 is provided by the followingformulas F13-F15.

D1=|A1−B2|=A1−B2   F13

D2=|A2−B1|=B1−A2   F14

D3=|A2−B2|=0   F15

The brightness of the final difference image at the first highbrightness portion is defined as E1, the brightness of the finaldifference image at the second high brightness portion is defined as E2,and the brightness of the final difference image at the other portionsis defined as E3. Each brightness E1-E3 is provided by the followingformulas F16-F18.

E1=C1−D1=(A1+B2)−(A1−B2)=2×B2   F16

E2=C2−D2=(A2+B1)−(B1−A2)=2×A2   F17

E3=C3−D3=(A2+B2)−0=A2+B2   F18

Thus, the brightness E1 of the final difference image at the first highbrightness portion can be replaced with the brightness B2 of the secondimage at the first high brightness portion. Further, the brightness E2of the final difference image at the second high brightness portion canbe replace with the brightness A2 of the first image at the second highbrightness portion.

As a result, the image information of the first image at the first highbrightness portion is replaced with the image information of the secondimage at the first high brightness portion. Further, the imageinformation of the second image at the second high brightness portion isreplaced with the image information of the first image at the secondhigh brightness portion. Thus, the brightness of the high brightnessportion is reproduced.

Alternatively, the controller may further include an addition imagegenerating means, an absolute difference image generating means and afinal difference image generating means. The addition image generatingmeans is configured to generate the addition image. The absolutedifference image generating means is configured to generate the absolutedifference image. The final difference image generating means isconfigured to generate the final difference image.

Alternatively, the image shooting element may be arranged at a middlepoint between the first light irradiation element and the second lightirradiation element. In this case, the difference between the firstimage of the object and the second image of the object is smaller than acase where the image shooting element is disposed near the first lightirradiation element or the second light irradiation element. Thus, theoverlapping portion of the first image of the object and the secondimage of the object is large. Thus, the region providing reproduction ofthe brightness of the high brightness portion becomes wide.

Alternatively, the controller may control the first and second lightirradiation elements to irradiate the first and second lightalternately, The controller controls the image shooting element to shootthe first image every time the first light irradiation elementirradiates the first light. The controller controls the image shootingelement to shoot the second image every time the second lightirradiation element irradiates the second light. The controllergenerates the addition image every time the controller generates alatest first image or the controller generates a latest second image.The controller generates the absolute difference image every time thecontroller generates the latest first image or the controller generatesthe latest difference image. In this case, the final difference imagefor providing reproduction of the brightness of the high brightnessportion is generated every time the first light irradiation elementirradiates the first light or the second light irradiation elementirradiates the second light. Further, the controller may generate thefinal difference image every time the controller generates a latestaddition image or the controller generates a latest absolute differenceimage.

Alternatively, the first and second light may be near infrared light.The controller further includes a non-irradiation image storing means,and the non-irradiation image storing means controls the image shootingelement to shoot a non-irradiation image only when no light irradiationelement irradiates light. In this case, the non-irradiation image hasinfluence of external light such as sunshine other than the nearinfrared light. Accordingly, a first difference image between the firstimage and the non-irradiation image and/or a second difference imagebetween the second image and the non-irradiation image are generated, sothat the first or second difference image has small influence of theexternal light other than the near infrared light.

Alternatively, the image shooting device may be mounted on a vehicle.The object is a head of a driver. The first light irradiation elementirradiates the first light toward a face of the driver, and the secondlight irradiation element irradiates the second light toward the face ofthe driver. In this case, the high brightness portion of the image ofthe face of the driver is highly reproduced. Thus, even if thebrightness of the eye of the driver in the image of the face of thedriver becomes high, the image information of the eye of the driver inthe image is reproduced. Thus, the face orientation and the eyedirection of the driver are detected with high accuracy.

Alternatively, the first light irradiation element, the second lightirradiation element and the image shooting element may be arranged on aninstrumental panel or adjacent to a steering column. Here, theinstrumental panel and the steering column are disposed in front of thedriver seat.

According to a second aspect of the present disclosure, a method forshooting an image of an object includes: irradiating first light;irradiating second light; receiving light and shooting an image withusing received light with an image shooting element; controlling theimage shooting element to shoot a first image of the object when thefirst light is irradiated; storing the first image; controlling theimage shooting element to shoot a second image of the object when thesecond light is irradiated; and storing the second image. The firstlight passes in a first irradiation region, and the second light passesin a second irradiation region. The first irradiation region ispartially overlapped with the second irradiation region. The imageshooting element is arranged at a predetermined position so that theimage shooting element receives light reflected on the object, the lightbeing one of the first and second lights. The object is arranged in anoverlapping region of the first and second irradiation regions.

In the above method, with using the first image and the second image,the high brightness portion is detected, and the image information atthe high brightness portion is reproduced without using a threshold.Thus, reliability of detection of the high brightness portion isimproved. Further, it is not necessary to replace the brightness at thehigh brightness portion with brightness of a periphery portion. Sincethe brightness at the high brightness portion is replaced withbrightness at the high brightness portion which is actually detected,reproducibility of the brightness at the high brightness portion isimproved.

Alternatively, the method may further include: generating an additionimage by adding brightness of the first image and brightness of thesecond image; generating an absolute difference image by calculating anabsolute value of difference between the brightness of the first imageand the brightness of the second image; and generating a finaldifference image by subtracting brightness of the absolute differenceimage from brightness of the addition image.

In the above case, the image information of the first image at the firsthigh brightness portion is replaced with the image information of thesecond image at the first high brightness portion. Further, the imageinformation of the second image at the second high brightness portion isreplaced with the image information of the first image at the secondhigh brightness portion. Thus, the brightness of the high brightnessportion is reproduced.

Alternatively, the irradiating the first light may be performed with afirst light irradiation element. The irradiating the second light isperformed with a second light irradiation element, and the imageshooting element is arranged at a middle point between the first lightirradiation element and the second light irradiation element.

Alternatively, the irradiating the first light and the irradiating thesecond light may be performed alternately. The controlling the imageshooting element to shoot the first image is performed every time thefirst light is irradiated. The controlling the image shooting element toshoot the second image is performed every time the second light isirradiated. The generating the addition image is performed every time alatest first image is generated or a latest second image is generated,and the generating the absolute difference image is performed every timethe latest first image is generated or the latest second image isgenerated. Further, the generating the final difference image may beperformed every time a latest addition image is generated or a latestabsolute difference image is generated.

Alternatively, the method may further include: controlling the imageshooting element to shoot a non-irradiation image of the object when nolight is irradiated; and storing the non-irradiation image. The firstand second light is a near infrared light. The controlling the imageshooting element to shoot the non-irradiation image and the storing thenon-irradiation image are performed by a non-irradiation image storingmeans, and the non-irradiation image storing means controls the imageshooting element to shoot the non-irradiation image only when no lightis irradiated.

Alternatively, the irradiating the first light may be performed by afirst light irradiation element, and the irradiating the second lightmay be performed by a second light irradiation element. The first lightirradiation element, the second light irradiation element and the imageshooting element are arranged on an instrumental panel or adjacent to asteering column.

According to a third aspect of the present disclosure, an image shootingdevice for shooting an image of an object includes: a first lightirradiation element for irradiating first light; a second lightirradiation element for irradiating second light; an image shootingelement for shooting an image with received light; and a controller. Thefirst light irradiation element has a first irradiation region, in whichthe first light from the first light irradiation element passes, and thesecond light irradiation element has a second irradiation region, inwhich the second light from the second light irradiation element passes.The first irradiation region is partially overlapped with the secondirradiation region. The image shooting element is disposed at apredetermined position so that the image shooting element receives lightreflected on the object, the light being one of the first and secondlight. The object is arranged in an overlapping region of the first andsecond irradiation regions. The controller controls the image shootingelement to shoot a non-irradiation image of the object when no lightirradiation element irradiates light. The controller stores thenon-irradiation image. The controller controls the image shootingelement to shoot a first image of the object when the first lightirradiation element irradiates the first light. The controller storesthe first image. The controller controls the image shooting element toshoot a second image of the object when the second light irradiationelement irradiates the second light, and the controller stores thesecond image.

In the above device, with using the first image and the second image,the high brightness portion is detected, and the image information atthe high brightness portion is reproduced without using a threshold.Thus, reliability of detection of the high brightness portion isimproved. Further, it is not necessary to replace the brightness at thehigh brightness portion with brightness of a periphery portion. Sincethe brightness at the high brightness portion is replaced withbrightness at the high brightness portion which is actually detected,reproducibility of the brightness at the high brightness portion isimproved. Further, the non-irradiation image has influence of externallight such as sunshine other than the near infrared light. Accordingly,a first difference image between the first image and the non-irradiationimage and/or a second difference image between the second image and thenon-irradiation image are generated, so that the first or seconddifference image has small influence of the external light other thanthe near infrared light.

Alternatively, the controller may generate a first difference image bysubtracting brightness of the non-irradiation image from brightness ofthe first image. The controller generates a second difference image bysubtracting the brightness of the non-irradiation image from brightnessof the second image. The controller generates an addition image byadding brightness of the first difference image and brightness of thesecond difference image. The controller generates an absolute differenceimage by calculating an absolute value of difference between thebrightness of the first difference image and the brightness of thesecond difference image, and the controller generates a third differenceimage by subtracting brightness of the absolute difference image frombrightness of the addition image. Further, the controller may controlthe first and second light irradiation elements to irradiate the firstand second light alternately. The controller may control the imageshooting element to shoot the non-irradiation image every time no lightirradiation element irradiates light. The controller may control theimage shooting element to shoot the first image every time the firstlight irradiation element irradiates the first light. The controller maycontrol the image shooting element to shoot the second image every timethe second light irradiation element irradiates the second light. Thecontroller generates the first difference image every time thecontroller stores a latest non-irradiation image or the controllerstores a latest first image. The controller generates the seconddifference image every time the controller stores a latestnon-irradiation image or the controller stores a latest second image.The controller generates the addition image every time the controllergenerates a latest first difference image or the controller generates alatest second difference image. The controller generates the absolutedifference image every time the controller generates the latest firstdifference image or the controller generates the latest seconddifference image. The controller generates the third difference imageevery time the controller generates a latest addition image or thecontroller generates a latest absolute difference image.

According to a fourth aspect of the present disclosure, a method forshooting an image of an object includes: irradiating first light;irradiating second light; receiving light and shooting an image withusing received light with an image shooting element; controlling theimage shooting element to shoot a non-irradiation image of the objectwhen no light is irradiated; storing the non-irradiation image;controlling the image shooting element to shoot a first image of theobject when the first light is irradiated; storing the first image;controlling the image shooting element to shoot a second image of theobject when the second light is irradiated; and storing the secondimage. The first light passes in a first irradiation region, and thesecond light passes in a second irradiation region. The firstirradiation region is partially overlapped with the second irradiationregion. The image shooting element is arranged at a predeterminedposition so that the image shooting element receives light reflected onthe object, the light being one of the first and second lights. Theobject is arranged in an overlapping region of the first and secondirradiation regions.

In the above method, with using the first image and the second image,the high brightness portion is detected, and the image information atthe high brightness portion is reproduced without using a threshold.Thus, reliability of detection of the high brightness portion isimproved. Further, it is not necessary to replace the brightness at thehigh brightness portion with brightness of a periphery portion. Sincethe brightness at the high brightness portion is replaced withbrightness at the high brightness portion which is actually detected,reproducibility of the brightness at the high brightness portion isimproved. Further, the non-irradiation image has influence of externallight such as sunshine other than the near infrared light. Accordingly,a first difference image between the first image and the non-irradiationimage and/or a second difference image between the second image and thenon-irradiation image are generated, so that the first or seconddifference image has small influence of the external light other thanthe near infrared light.

Alternatively, the method may further include: generating a firstdifference image by subtracting brightness of the non-irradiation imagefrom brightness of the first image; generating a second difference imageby subtracting the brightness of the non-irradiation image frombrightness of the second image; generating an addition image by addingbrightness of the first difference image and brightness of the seconddifference image; generating an absolute difference image by calculatingan absolute value of difference between the brightness of the firstdifference image and the brightness of the second difference image; andgenerating a third difference image by subtracting brightness of theabsolute difference image from brightness of the addition image.Further, the irradiating the first light and the irradiating the secondlight may be performed alternately. The controlling the image shootingelement to shoot the non-irradiation image is performed every time nolight is irradiated. The controlling the image shooting element to shootthe first image is performed every time the first light is irradiated.The controlling the image shooting element to shoot the second image isperformed every time the second light is irradiated. The generating thefirst difference image is performed every time a latest non-irradiationimage is stored or a latest first image is stored. The generating thesecond difference image is performed every time a latest non-irradiationimage is stored or a latest second image is stored. The generating theaddition image is performed every time a latest first difference imageis generated or a latest second difference image is generated. Thegenerating the absolute difference image is performed every time thelatest first difference image is generated or the latest seconddifference image is generated, and the generating the third differenceimage is performed every time a latest addition image is generated or alatest absolute difference image is generated.

While the invention has been described with reference to preferredembodiments thereof, it is to be understood that the invention is notlimited to the preferred embodiments and constructions. The invention isintended to cover various modification and equivalent arrangements. Inaddition, while the various combinations and configurations, which arepreferred, other combinations and configurations, including more, lessor only a single element, are also within the spirit and scope of theinvention.

1. An image shooting device for shooting an image of an objectcomprising: a first light irradiation element for irradiating firstlight; a second light irradiation element for irradiating second light;an image shooting element for shooting an image with received light; anda controller, wherein the first light irradiation element has a firstirradiation region, in which the first light from the first lightirradiation element passes, and the second light irradiation element hasa second irradiation region, in which the second light from the secondlight irradiation element passes, wherein the first irradiation regionis partially overlapped with the second irradiation region, wherein theimage shooting element is disposed at a predetermined position so thatthe image shooting element receives light reflected on the object, thelight being one of the first and second light, wherein the object isarranged in an overlapping region of the first and second irradiationregions, wherein the controller controls the image shooting element toshoot a first image of the object when the first light irradiationelement irradiates the first light, wherein the controller stores thefirst image, wherein the controller controls the image shooting elementto shoot a second image of the object when the second light irradiationelement irradiates the second light, and wherein the controller storesthe second image.
 2. The image shooting device according to claim 1,wherein the controller includes a first light irradiation control means,a first image storing means, a second light irradiation control meansand a second image storing means, wherein the first light irradiationcontrol means controls the first light irradiation element to irradiateonly the first light, wherein the first image storing means controls theimage shooting element to shoot the first image only when the firstlight irradiation element irradiates the first light, wherein the secondlight irradiation control means controls the second light irradiationelement to irradiate only the second light, and wherein the second imagestoring means controls the image shooting element to shoot the secondimage only when the second light irradiation element irradiates thesecond light.
 3. The image shooting device according to claim 1, whereinthe controller generates an addition image by adding brightness of thefirst image and brightness of the second image, wherein the controllergenerates an absolute difference image by calculating an absolute valueof difference between the brightness of the first image and thebrightness of the second image, and wherein the controller generates afinal difference image by subtracting brightness of the absolutedifference image from brightness of the addition image.
 4. The imageshooting device according to claim 3, wherein the controller furtherincludes an addition image generating means, an absolute differenceimage generating means and a final difference image generating means,wherein the addition image generating means is configured to generatethe addition image, wherein the absolute difference image generatingmeans is configured to generate the absolute difference image, andwherein the final difference image generating means is configured togenerate the final difference image.
 5. The image shooting deviceaccording to claim 1, wherein the image shooting element is arranged ata middle point between the first light irradiation element and thesecond light irradiation element.
 6. The image shooting device accordingto claim 3, wherein the controller controls the first and second lightirradiation elements to irradiate the first and second lightalternately, wherein the controller controls the image shooting elementto shoot the first image every time the first light irradiation elementirradiates the first light, wherein the controller controls the imageshooting element to shoot the second image every time the second lightirradiation element irradiates the second light, wherein the controllergenerates the addition image every time the controller generates alatest first image or the controller generates a latest second image,and wherein the controller generates the absolute difference image everytime the controller generates the latest first image or the controllergenerates the latest difference image.
 7. The image shooting deviceaccording to claim 6, wherein the controller generates the finaldifference image every time the controller generates a latest additionimage or the controller generates a latest absolute difference image. 8.The image shooting device according to claim 1, wherein the first andsecond light is near infrared light, wherein the controller furtherincludes a non-irradiation image storing means, and wherein thenon-irradiation image storing means controls the image shooting elementto shoot a non-irradiation image only when no light irradiation elementirradiates light.
 9. The image shooting device according to claim 1,wherein the image shooting device is mounted on a vehicle, wherein theobject is a head of a driver, wherein the first light irradiationelement irradiates the first light toward a face of the driver, andwherein the second light irradiation element irradiates the second lighttoward the face of the driver.
 10. The image shooting device accordingto claim 9, wherein the first light irradiation element, the secondlight irradiation element and the image shooting element are arranged onan instrumental panel or adjacent to a steering column.
 11. A method forshooting an image of an object comprising: irradiating first light;irradiating second light; receiving light and shooting an image withusing received light with an image shooting element; controlling theimage shooting element to shoot a first image of the object when thefirst light is irradiated; storing the first image; controlling theimage shooting element to shoot a second image of the object when thesecond light is irradiated; and storing the second image, wherein thefirst light passes in a first irradiation region, and the second lightpasses in a second irradiation region, wherein the first irradiationregion is partially overlapped with the second irradiation region,wherein the image shooting element is arranged at a predeterminedposition so that the image shooting element receives light reflected onthe object, the light being one of the first and second lights, andwherein the object is arranged in an overlapping region of the first andsecond irradiation regions.
 12. The method according to claim 11,further comprising: generating an addition image by adding brightness ofthe first image and brightness of the second image; generating anabsolute difference image by calculating an absolute value of differencebetween the brightness of the first image and the brightness of thesecond image; and generating a final difference image by subtractingbrightness of the absolute difference image from brightness of theaddition image.
 13. The method according to claim 11, wherein theirradiating the first light is performed with a first light irradiationelement, wherein the irradiating the second light is performed with asecond light irradiation element, and wherein the image shooting elementis arranged at a middle point between the first light irradiationelement and the second light irradiation element.
 14. The methodaccording to claim 12, wherein the irradiating the first light and theirradiating the second light are performed alternately, wherein thecontrolling the image shooting element to shoot the first image isperformed every time the first light is irradiated, wherein thecontrolling the image shooting element to shoot the second image isperformed every time the second light is irradiated, wherein thegenerating the addition image is performed every time a latest firstimage is generated or a latest second image is generated, and whereinthe generating the absolute difference image is performed every time thelatest first image is generated or the latest second image is generated.15. The method according to claim 14, wherein the generating the finaldifference image is performed every time a latest addition image isgenerated or a latest absolute difference image is generated.
 16. Themethod according to claim 11, further comprising: controlling the imageshooting element to shoot a non-irradiation image of the object when nolight is irradiated; and storing the non-irradiation image; wherein thefirst and second light is a near infrared light, wherein the controllingthe image shooting element to shoot the non-irradiation image and thestoring the non-irradiation image are performed by a non-irradiationimage storing means, and wherein the non-irradiation image storing meanscontrols the image shooting element to shoot the non-irradiation imageonly when no light is irradiated.
 17. The method according to claim 11,wherein the image shooting element is mounted on a vehicle, wherein theobject is a head of a driver, wherein the first light is irradiatedtoward a face of the driver, and wherein the second light is irradiatedtoward the face of the driver.
 18. The image shooting device accordingto claim 17, wherein the irradiating the first light is performed by afirst light irradiation element, wherein the irradiating the secondlight is performed by a second light irradiation element, wherein thefirst light irradiation element, the second light irradiation elementand the image shooting element are arranged on an instrumental panel oradjacent to a steering column.
 19. An image shooting device for shootingan image of an object comprising: a first light irradiation element forirradiating first light; a second light irradiation element forirradiating second light; an image shooting element for shooting animage with received light; and a controller, wherein the first lightirradiation element has a first irradiation region, in which the firstlight from the first light irradiation element passes, and the secondlight irradiation element has a second irradiation region, in which thesecond light from the second light irradiation element passes, whereinthe first irradiation region is partially overlapped with the secondirradiation region, wherein the image shooting element is disposed at apredetermined position so that the image shooting element receives lightreflected on the object, the light being one of the first and secondlight, wherein the object is arranged in an overlapping region of thefirst and second irradiation regions, wherein the controller controlsthe image shooting element to shoot a non-irradiation image of theobject when no light irradiation element irradiates light, wherein thecontroller stores the non-irradiation image, wherein the controllercontrols the image shooting element to shoot a first image of the objectwhen the first light irradiation element irradiates the first light,wherein the controller stores the first image, wherein the controllercontrols the image shooting element to shoot a second image of theobject when the second light irradiation element irradiates the secondlight, and wherein the controller stores the second image.
 20. The imageshooting device according to claim 19, wherein the controller generatesa first difference image by subtracting brightness of thenon-irradiation image from brightness of the first image, wherein thecontroller generates a second difference image by subtracting thebrightness of the non-irradiation image from brightness of the secondimage, wherein the controller generates an addition image by addingbrightness of the first difference image and brightness of the seconddifference image, wherein the controller generates an absolutedifference image by calculating an absolute value of difference betweenthe brightness of the first difference image and the brightness of thesecond difference image, and wherein the controller generates a thirddifference image by subtracting brightness of the absolute differenceimage from brightness of the addition image.
 21. The image shootingdevice according to claim 20, wherein the controller controls the firstand second light irradiation elements to irradiate the first and secondlight alternately, wherein the controller controls the image shootingelement to shoot the non-irradiation image every time no lightirradiation element irradiates light, wherein the controller controlsthe image shooting element to shoot the first image every time the firstlight irradiation element irradiates the first light, wherein thecontroller controls the image shooting element to shoot the second imageevery time the second light irradiation element irradiates the secondlight, wherein the controller generates the first difference image everytime the controller stores a latest non-irradiation image or thecontroller stores a latest first image, wherein the controller generatesthe second difference image every time the controller stores a latestnon-irradiation image or the controller stores a latest second image,wherein the controller generates the addition image every time thecontroller generates a latest first difference image or the controllergenerates a latest second difference image, wherein the controllergenerates the absolute difference image every time the controllergenerates the latest first difference image or the controller generatesthe latest second difference image, and wherein the controller generatesthe third difference image every time the controller generates a latestaddition image or the controller generates a latest absolute differenceimage.
 22. A method for shooting an image of an object comprising:irradiating first light; irradiating second light; receiving light andshooting an image with using received light with an image shootingelement; controlling the image shooting element to shoot anon-irradiation image of the object when no light is irradiated; storingthe non-irradiation image; controlling the image shooting element toshoot a first image of the object when the first light is irradiated;storing the first image; controlling the image shooting element to shoota second image of the object when the second light is irradiated; andstoring the second image, wherein the first light passes in a firstirradiation region, and the second light passes in a second irradiationregion, wherein the first irradiation region is partially overlappedwith the second irradiation region, wherein the image shooting elementis arranged at a predetermined position so that the image shootingelement receives light reflected on the object, the light being one ofthe first and second lights, and wherein the object is arranged in anoverlapping region of the first and second irradiation regions.
 23. Themethod according to claim 22, further comprising: generating a firstdifference image by subtracting brightness of the non-irradiation imagefrom brightness of the first image, generating a second difference imageby subtracting the brightness of the non-irradiation image frombrightness of the second image; generating an addition image by addingbrightness of the first difference image and brightness of the seconddifference image; generating an absolute difference image by calculatingan absolute value of difference between the brightness of the firstdifference image and the brightness of the second difference image; andgenerating a third difference image by subtracting brightness of theabsolute difference image from brightness of the addition image.
 24. Themethod according to claim 23, wherein the irradiating the first lightand the irradiating the second light are performed alternately, whereinthe controlling the image shooting element to shoot the non-irradiationimage is performed every time no light is irradiated, wherein thecontrolling the image shooting element to shoot the first image isperformed every time the first light is irradiated, wherein thecontrolling the image shooting element to shoot the second image isperformed every time the second light is irradiated, wherein thegenerating the first difference image is performed every time a latestnon-irradiation image is stored or a latest first image is stored,wherein the generating the second difference image is performed everytime a latest non-irradiation image is stored or a latest second imageis stored, wherein the generating the addition image is performed everytime a latest first difference image is generated or a latest seconddifference image is generated, wherein the generating the absolutedifference image is performed every time the latest first differenceimage is generated or the latest second difference image is generated,and wherein the generating the third difference image is performed everytime a latest addition image is generated or a latest absolutedifference image is generated.